Reproducible
data analysis

Reproducible research in a context

Providing a reproducible analysis is the most important and easiest aspect of open science

• data cannot always be easily shared
• analysis connects data and results
• confirming pre-registered hypotheses is less valuable without it
• pre-registration is often not possible or useful ¹

Why reproducible research?

• builds trust
• reduces errors
• makes it easier to write the method section ¹
• streamlines manuscript writing

Levels of reproducible data analysis

Tools for reproducible data analysis

1. Scripting

Different scripts for different steps (Example)

Working around slow analysis parts

Save intermediate steps

• pre-processing
• slow model estimation (MCMC)

fn = "my_analysis_results.Rdata"

if (file.exists(fn)) {
  load(fn)
} else {
  # make my analysis
  save(my_fit, file = fn)
}

2. Version control

Start each paper by setting up a (git/github) project

• easily allows to go back to older versions of analyses
• saves as backup method ¹
• allows to work jointly on a project
• is well integrated with git/github (show)
• first step to later publishing the code
• Hard: Sufficient & continuous code documentation

3. Analysis pipeline in R Markdown

• R Markdown document¹ for
  • data cleaning
  • statistical analysis
  • supplementary
    analyses & results
  • generation of
    statistics for paper

4. Writing papers with papaja

on CRAN & github

• APA-6 compliant manuscripts via R Markdown
• GUI to search references (based on bibtex files)
• automatic references to figures & tables in the supplement
• APA styles for tables and figures
• outputs MS Word or PDF files
• Upfront investment, but provides huge benefits!

Example

Things I didn’t talk about

• testing code: blog, video

• code documentation: blog, editorial

• workflow management packages targets, workflowr

• Should I write a package?

Summary